Proxy-pattern-based multimedia displaying method, device and apparatus

ABSTRACT

A proxy-pattern-based multimedia displaying method is disclosed in the present disclosure, the multimedia displaying method is applied to a chip where an embedded system is built in. The embedded system includes a display application layer, a surface proxy body, a surface entrusting body and a surface realizing component. The surface proxy body is an proxy for the surface entrusting body. The surface proxy body and the surface entrusting body are independent of the display application layer. The multimedia displaying method includes: receiving to-be-displayed multimedia data from the display application layer and sending the to-be-displayed multimedia data to the surface entrusting body by the surface proxy body; and sending the to-be-displayed multimedia data to the surface realizing component or the display application layer for display on a display screen by the surface entrusting body.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims priorities to Chinese Patent Application No.201810278774.9, filed on Mar. 30, 2018 in the State IntellectualProperty Office of China, the contents of which are herein incorporatedby reference in their entireties.

TECHNICAL FIELD

The present disclosure generally relates to the technical field ofdisplaying, and more particularly, to a proxy-pattern-based multimediadisplaying method, device and a readable storage medium.

BACKGROUND

An operating system generally comprises a surface realizing component(which comprises a display drive or a display system) therein to providea uniform interface for accessing video apparatuses/drives at a lowerlevel by a surface application program at an upper level. The surfaceapplication program may call the surface realizing component to realizethe display of contents such as videos, pictures and subtitle. For afile that needs to be decoded, the application program may assign thefile to a middleware (e.g., a play framework, a picture displayframework or the like) to be decoded by the middleware and then thedecoded file is displayed by calling the surface realizing component.

In the prior art, a display application layer (the surface applicationprogram and/or the middleware) directly calls the surface realizingcomponent. For example, in a Linux system, a drive program V412 providesa set of interface specifications for video apparatus programs underLinux. Chip manufacturers can achieve the interface of a V412 drive anda display apparatus according to the interface definition. The displayapplication layer can achieve the display of videos, subtitle andpictures or the like by calling a Video-associated interface of the V412drive. A display control part in the display application layer needs todirectly interface with the surface realizing component. If the surfacerealizing component needs to be replaced, or interfaces/parameterconfiguration of the surface realizing component vary (e.g., beingimplanted onto different chips), then the display application layerneeds to be modified with a high complexity and a poor flexibility.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to make the technical solution described in the embodiments ofthe present disclosure more clear, the drawings used for the descriptionof the embodiments will be briefly described. Apparently, the drawingsdescribed below are only for illustration but not for limitation. Itshould be understood that, one skilled in the art may acquire otherdrawings based on these drawings, without making any inventive work.

FIG. 1 is a schematic flowchart diagram of an embodiment of aproxy-pattern-based multimedia displaying method according to thepresent disclosure.

FIG. 2 is a schematic flowchart diagram of S1 in the case whereto-be-displayed multimedia data is transmitted in the form of a bufferin an embodiment of the proxy-pattern-based multimedia displaying methodaccording to the present disclosure.

FIG. 3 is a schematic view illustrating the overall architecture in anembodiment of the proxy-pattern-based multimedia displaying methodaccording to the present disclosure.

FIG. 4 is a schematic view illustrating the overall architecture in thecase where a surface entrusting body is of a non-CS architecture anddoes not comprise a surface realizing component in an embodiment of theproxy-pattern-based multimedia displaying method according to thepresent disclosure.

FIG. 5 is a schematic view illustrating the overall architecture in thecase where the surface entrusting body is of a CS architecture and doesnot comprise the surface realizing component in an embodiment of theproxy-pattern-based multimedia displaying method according to thepresent disclosure.

FIG. 6 is a schematic view illustrating the overall architecture in thecase where the surface entrusting body is of a non-CS architecture andcomprises the surface realizing component in an embodiment of theproxy-pattern-based multimedia displaying method according to thepresent disclosure.

FIG. 7 is a schematic view illustrating the overall architecture in thecase where the surface entrusting body is of a CS architecture andcomprises the surface realizing component in an embodiment of theproxy-pattern-based multimedia displaying method according to thepresent disclosure.

FIG. 8 is a schematic view illustrating that one surface applicationprogram corresponds to one display screen in an embodiment of theproxy-pattern-based multimedia displaying method according to thepresent disclosure.

FIG. 9 is a schematic view illustrating that to-be-displayed multimediadata from one surface application program is displayed on twocorresponding display screens simultaneously in an embodiment of theproxy-pattern-based multimedia displaying method according to thepresent disclosure.

FIG. 10 is a schematic view illustrating that to-be-displayed multimediadata from two surface application programs is displayed on onecorresponding display screen simultaneously in an embodiment of theproxy-pattern-based multimedia displaying method according to thepresent disclosure.

FIG. 11 is a schematic view of selecting to-be-displayed multimedia datathat can be displayed according to priority in the case where the numberof display screens cannot satisfy the displaying need of the surfaceapplication program in an embodiment of the proxy-pattern-basedmultimedia displaying method according to the present disclosure.

FIG. 12 is an overall schematic displaying flowchart diagram of aspecific embodiment of the proxy-pattern-based multimedia displayingmethod according to the present disclosure.

FIG. 13 is a schematic flowchart diagram in the case where the surfaceentrusting body is of a non-CS architecture and processes a control flowin a specific embodiment of the proxy-pattern-based multimediadisplaying method according to the present disclosure.

FIG. 14 is a schematic flowchart diagram in the case where the surfaceentrusting body is of a non-CS architecture and processes a data flow ina specific embodiment of the proxy-pattern-based multimedia displayingmethod according to the present disclosure.

FIG. 15 is a schematic flowchart diagram in the case where the surfaceentrusting body is of a CS architecture and processes a control flow ina specific embodiment of the proxy-pattern-based multimedia displayingmethod according to the present disclosure.

FIG. 16 is a schematic flowchart diagram in the case where the surfaceentrusting body is of a CS architecture and processes a data flow in aspecific embodiment of the proxy-pattern-based multimedia displayingmethod according to the present disclosure.

FIG. 17 is an overall schematic displaying flowchart diagram of anotherspecific embodiment of the proxy-pattern-based multimedia displayingmethod according to the present disclosure.

FIG. 18 is a schematic flowchart diagram in the case where the surfaceentrusting body is of a non-CS architecture and processes a control flowin another specific embodiment of the proxy-pattern-based multimediadisplaying method according to the present disclosure.

FIG. 19 is a schematic flowchart diagram in the case where the surfaceentrusting body is of a non-CS architecture and processes a data flow inanother specific embodiment of the proxy-pattern-based multimediadisplaying method according to the present disclosure.

FIG. 20 is an overall schematic displaying flowchart diagram of yetanother specific embodiment of the proxy-pattern-based multimediadisplaying method according to the present disclosure.

FIG. 21 is an overall schematic displaying flowchart diagram of afurther specific embodiment of the proxy-pattern-based multimediadisplaying method according to the present disclosure.

FIG. 22 is a schematic flowchart diagram in the case where the surfaceentrusting body is of a non-CS architecture and processes a control flowin a further specific embodiment of the proxy-pattern-based multimediadisplaying method according to the present disclosure.

FIG. 23 is a schematic flowchart diagram in the case where the surfaceentrusting body is of a non-CS architecture and processes a data flow ina further specific embodiment of the proxy-pattern-based multimediadisplaying method according to the present disclosure.

FIG. 24 is a schematic flowchart diagram in the case where the surfaceentrusting body is of a CS architecture and processes a control flow ina further specific embodiment of the proxy-pattern-based multimediadisplaying method according to the present disclosure.

FIG. 25 is a schematic flowchart diagram in the case where the surfaceentrusting body is of a CS architecture and processes a data flow in afurther specific embodiment of the proxy-pattern-based multimediadisplaying method according to the present disclosure.

FIG. 26 is a schematic structural view of an embodiment of aproxy-pattern-based multimedia displaying device according to thepresent disclosure.

FIG. 27 is a schematic structural view of an embodiment of theproxy-pattern-based multimedia displaying apparatus according to thepresent disclosure.

FIG. 28 is a schematic structural view of an embodiment of a readablestorage medium according to the present disclosure.

DETAILED DESCRIPTION

The present disclosure will be described in detail with reference toattached drawings and embodiments hereinafter. Contents not conflictingin the following embodiments may be combined with each other.

As shown in FIG. 1, an embodiment of a proxy-pattern-based multimediadisplaying method according to the present disclosure comprises:

S1: receiving to-be-displayed multimedia data from the displayapplication layer and sending the to-be-displayed multimedia data to thesurface entrusting body by the surface proxy body.

This embodiment is applied to a chip where an embedded system is builtin, e.g., a vehicle-mounted display chip where a Linux system is builtin. The embedded system comprises a display application layer, a surfaceproxy body, a surface entrusting body and a surface realizing component.The display application layer comprises a surface application programand/or a middleware, and the surface realizing component comprises adisplay drive or a display system. The surface proxy body and thesurface entrusting body are independent of the display applicationlayer.

The surface application program may be an application program thatrequires the display function, e.g., a video playing application, apicture browsing application or the like. Contents to be displayed(videos/subtitle/pictures or the like) may be encoded by a certain kindof encoding manner, so the surface application program may need todecode the encoded contents before displaying the encoded contents. Thedecoding operation may be accomplished by the surface applicationprogram itself or be accomplished by calling a middleware. For videofiles, in addition to being decoded, the video files also need to bedivided so as to separate video streams, audio streams and possiblyincluded subtitle streams from each other. Similarly the dividing mayalso be accomplished by the surface application program itself or beaccomplished by calling a middleware. The middleware may comprise atleast one of a play framework, a picture display framework or the like.

The play framework (e.g., Gstreamer, DirectShow, ffmpeg or the like) maybe construed as a software platform, and a set of intuitive and compactAPI interfaces are designed on the platform. The upper-level applicationprogram can transparently use functions provided by the softwareplatform through these interfaces, e.g., file dividing, parsing,encoding and decoding or the like.

The surface proxy body provides a plurality of interfaces, that can beused for realizing the display function, to be called by the displayapplication layer, but the surface proxy body itself does not have thedisplay function, and the actual display function is realized by thesurface entrusting body. To realize the display function, the surfaceentrusting body needs to interface with the surface realizing component,or the surface entrusting body comprises the surface realizing componentso as to internally interface with the surface realizing component. Thesurface proxy body, as the proxy of the surface entrusting body, ismainly responsible for interaction of data and control commands betweenthe display application layer and the surface entrusting body.

Optionally, before this step, the surface proxy body may receiveconfiguration information from the display application layer and forwardthe configuration information to the surface entrusting body. Thetransmission of the configuration information may be accomplished at onetime or at least at two times. The configuration information comprisesat least one of a selected format, resolution, buffer information,destination information and priority. The buffer information comprisesformats of the buffers and/or the count of the buffers, and thedestination information comprises at least one of a destinationposition, a destination width, and a destination height.

Default values of part or all of the configuration information may beagreed upon in advance between the display application layer and thesurface entrusting body/surface realizing component. If the values ofsome kinds of configuration information determined by the displayapplication layer are the default values, then these kinds ofconfiguration information may be selected to be omitted during theconfiguration process.

Optionally, before the transmission of the configuration information,the display application layer may acquire supported attributeinformation of the surface SDK through the surface proxy body, and thesupported attribute information comprises a set of formats supportedand/or a set of characters supported. Generally speaking, the set offormats supported is for displayed contents of a picture attribute,e.g., videos, pictures, subtitle in a picture form or the like; and theset of characters supported is for displayed contents of a textattribute, e.g., characters, subtitle in a text form or the like.Additionally, the supported attribute information may also comprisesupported information of priority, supported information of multiplexdisplaying or the like.

The to-be-displayed multimedia data may comprise at least one ofsubtitle, decoded picture data, and decoded video data. Theto-be-displayed multimedia data being transmitted may be itself or abuffer filled with the to-be-displayed multimedia data by the displayapplication layer.

As shown in FIG. 2, if the to-be-displayed multimedia data istransmitted in the form of a buffer, then S1 may specifically comprise:

S11: acquiring by the surface proxy body a output buffer from thesurface entrusting body.

S12: sending by the surface proxy body the output buffer to the displayapplication layer so that the display application layer fills the outputbuffer with the to-be-displayed multimedia data.

If the to-be-displayed multimedia data comprises text subtitle, i.e.,subtitle in the text form, then the display application layer firstneeds to convert the text subtitle into subtitle of a picture form andthen fills the output buffer with the to-be-displayed multimedia data.

S13: receiving by the surface proxy body the filled buffer from thedisplay application layer.

S14: sending by the surface proxy body the filled buffer to the surfaceentrusting body.

According to the actual architecture of the surface entrusting body, thesurface proxy body may send the filled buffer to a surface realizationor a surface client.

The surface entrusting body/surface realizing component may read data inthe filled buffer for display, and the buffer becomes free again afterthe reading/displaying is finished.

S2: sending by the surface entrusting body the to-be-displayedmultimedia data to the surface realizing component or the displayapplication layer for display on a display screen.

Generally speaking, the surface entrusting body may call the surfacerealizing component to plot the to-be-displayed multimedia data, therebyrealizing the display of the to-be-displayed multimedia data on thedisplay screen. In some embodiments, the surface entrusting body maycall other programs to display the to-be-displayed multimedia data. Forexample, when the to-be-displayed multimedia data comprises thesubtitle, the surface entrusting body may call a function provided bythe surface application program to plot the subtitle.

Through the implementation of this embodiment, the display applicationlayer receives the to-be-displayed multimedia data from the displayapplication layer and sends the to-be-displayed multimedia data to thesurface entrusting body through the surface proxy body; the surfaceentrusting body sends the to-be-displayed multimedia data to the surfacerealizing component or the display application layer for display on adisplay screen. It is the surface entrusting body that interfaces withthe surface realizing component or the display application layer torealize the display function, the surface proxy body is a proxy for thesurface entrusting body, the display application layer indirectly callsthe surface entrusting body through the surface proxy body and does notinterface with the surface entrusting body directly and thus does notinterface with the surface realizing component directly. Therefore, evenif the surface realizing component varies, e.g., being implanted todifferent chips, it is unnecessary to modify the display applicationlayer, thereby reducing the complexity of the display application layerand improving the flexibility of the display application layer.

An overall architecture of this embodiment is as shown in FIG. 3, and adisplay application layer 11 calls a surface entrusting body 13 througha surface proxy body 12. Because the display application layer 11 doesnot directly interface with the surface entrusting body 13, the surfaceentrusting body 13 actually realizing the display function may be of aclient/server (CS) architecture. Depending on whether the surfaceentrusting body 13 is of a CS architecture and whether the surfaceentrusting body 13 comprises the surface realizing component therein,the overall architecture shown in FIG. 3 may be divided into fourcategories, which will be described with reference to attached drawingshereinafter.

In the case where the surface entrusting body is of a non-CSarchitecture and does not comprise the surface realizing component, theoverall architecture is as shown in FIG. 4, wherein a surface entrustingbody 130 comprises a surface realization 131, and the surfacerealization 131 is configured to interface with a surface realizingcomponent 140 to realize the display function. Specifically, the surfacerealization 131 may send the to-be-displayed multimedia data from asurface proxy body 120 to a surface realizing component 140 so that thesurface realizing component 140 controls a display screen 150 to displaythe to-be-displayed multimedia data. Optionally, the surface entrustingbody 130 and the surface proxy body 120 may belong to the same processas a display application layer 110.

In the case where the surface entrusting body is the CS architecture anddoes not comprise the surface realizing component, the overallarchitecture is as shown in FIG. 5, wherein a surface entrusting body230 comprises a surface client 231 and a surface server 232, and thesurface server 232 is configured to interface with a surface realizingcomponent 240 to realize the display function, and the surface client231 is configured to interface with a surface proxy body 220.Specifically, the surface client 231 may send the to-be-displayedmultimedia data from the surface proxy body 220 to the surface server232, and the surface server 232 may send the received to-be-displayedmultimedia data to the surface realizing component 240 so that thesurface realizing component 240 controls a display screen 250 to displaythe to-be-displayed multimedia data. Optionally, the surface client 231and the surface server 232 may belong to different processes, andinteract with each other through inter-process communication.

In the case where the surface entrusting body is the non-CS architectureand comprises the surface realizing component, the overall architectureis as shown in FIG. 6, and the architecture differs from thearchitecture shown in FIG. 4 in that: a surface entrusting body 330further comprises a surface realizing component 332, and reference maybe made to the above corresponding description for specific contentsthereof.

In the case where the surface entrusting body is the CS architecture andcomprises the surface realizing component, the overall architecture isas shown in FIG. 7, and the architecture differs from the architectureshown in FIG. 5 in that: a surface entrusting body 430 further comprisesa surface realizing component 433, and reference may be made to theabove corresponding description for specific contents thereof.

If the number of surface application programs and/or display screensthat require displaying at the same time is greater than one (which iscalled multiplex displaying for short), each surface application programcorresponds to one surface proxy body, and for the surface entrustingbody, each surface application program needs to correspond to onesurface realization respectively under the non-CS architecture, eachsurface application program needs to correspond to one surface clientrespectively under the CS architecture, each surface client correspondsto one surface object respectively in the surface server, and thesurface server manages all the surface objects. For the non-CSarchitecture, the surface realization directly interfaces with thesurface realizing component, so the control of multiplex displayingneeds to be accomplished by the surface realizing component, whichincreases the complexity of the surface realizing component and makes ithard to control the surface realizing component. For the CSarchitecture, the surface server manages all the surface objects and iscapable of acquiring the configuration information and to-be-displayedmultimedia data of each surface object, thereby determining whether todisplay and how to display the to-be-displayed multimedia data of eachsurface object and accomplishing the control of multiplex displaying.Because the control of multiplex displaying is accomplished by thesurface server rather than by the surface realizing component under theCS architecture, it is unnecessary to use complex surface realizingcomponent and the processing of multiplex displaying is more convenientthan that under the non-CS architecture.

Specifically, multiplex displaying may be divided into multiple casesdepending on the number of the surface application programs and thedisplay screens as well as displaying need of the surface applicationprograms, and examples thereof will be illustrated with reference to theattached drawings hereinafter.

One case is that one surface application program corresponds to onedisplay screen for display. For example, referring to FIG. 8, a surfaceapplication program A sends to-be-displayed multimedia data a to asurface object a in the surface server through a surface proxy body aand a surface client a, a surface application program B sendsto-be-displayed multimedia data b to a surface object b in the surfaceserver through a surface proxy body b and a surface client b, thesurface server sends the to-be-displayed multimedia data a to a coverageobject 1 (corresponding to a display screen 1) in the display system fordisplay on the display screen 1, and sends the to-be-displayedmultimedia data b to a coverage object 2 (corresponding to a displayscreen 2) in the display system for display on the display screen 2.

Another case is that one surface application program respectivelycorresponds to at least two display screens for display, i.e., theto-be-displayed multimedia data from one surface application program isdisplayed on at least two corresponding display screens simultaneously,and this need may be satisfied by copying the to-be-displayed multimediadata. For example, referring to FIG. 9, a surface application program Csends to-be-displayed multimedia data c to a surface object c in thesurface server through a surface proxy body c and a surface client c,the surface server copies the to-be-displayed multimedia data c toobtain to-be-displayed multimedia data c′, sends the to-be-displayedmultimedia data c to a coverage object 3 (corresponding to a displayscreen 3) in the display system for display on the display screen 3, andsends the to-be-displayed multimedia data c′ to a coverage object 4(corresponding to a display screen 4) in the display system for displayon the display screen 4.

Yet another case is that at least two surface application programscorrespond to one display screen, i.e., the to-be-displayed multimediadata from two surface application programs is displayed on onecorresponding display screen simultaneously, and this need may besatisfied by mixing the to-be-displayed multimedia data. For example,referring to FIG. 10, a surface application program D sendsto-be-displayed multimedia data d to a surface object d in the surfaceserver through a surface proxy body d and a surface client d, a surfaceapplication program E sends to-be-displayed multimedia data e to asurface object e in the surface server through a surface proxy body eand a surface client e, and the surface server mixes and sends theto-be-displayed multimedia data d and the to-be-displayed multimediadata e to a coverage object 5 (corresponding to a display screen 5) inthe display system for simultaneous display on the display screen 5.

In the ideal case, destinations occupied by the to-be-displayedmultimedia data d and the to-be-displayed multimedia data e are notoverlapped with each other and the two destinations can be displayedcompletely at the same time. But actually, the two destinations may beoverlapped with each other, and in this case, which destination is atthe front (i.e., displayed at the overlapped portion) and whichdestination is at the back (i.e., not displayed at the overlappedportion) may be determined by the surface server, e.g., determineddepending on the priority of the surface application programs D and E,and the destination of a higher priority is at the front.

The number of the display screens needed by each surface applicationprogram is not necessarily one, so it may be first determined whetherthe number of the display screens satisfies the displaying need of thesurface application programs when multiple surface application programsneed to be displayed. If the number of the display screens cannotsatisfy the displaying need of the surface application program, then theto-be-displayed multimedia data of a part of the surface applicationprograms cannot be displayed. For example, referring to FIG. 11, asurface application program F sends to-be-displayed multimedia data f toa surface object f in the surface server through a surface proxy body fand a surface client f, a surface application program G sendsto-be-displayed multimedia data g to a surface object g in the surfaceserver through a surface proxy body g and a surface client g. Becausethere is only one display screen 6, and the surface application programF requires full screen display, the surface server cannot mix theto-be-displayed multimedia data f and g and can only select and send oneof the to-be-displayed multimedia data f and g to a coverage object 6(corresponding to the display screen 6) for display. In this example,the to-be-displayed multimedia data f from the surface applicationprogram F of a higher priority is selected, and the to-be-displayedmultimedia data g is not displayed.

Although four cases are illustrated in the above descriptions, the fourcases are not mutually exclusive but may co-occur in combination. In theabove examples, except for the single surface applicationprogram/display screen particularly defined, the number of other surfaceapplication programs/display screens (e.g., two) is only forillustration and the actual number may vary as long as the limitcondition (e.g., at least two) is satisfied; it is the display systemthat interfaces with the display screen in all of the above examples,but actually what interfaces with the display screen may be a displaydrive. Moreover, in all of the above examples, the surface entrustingbody is based on the CS architecture, but actually the multiplexdisplaying may also be applied to the surface entrusting body based onthe non-CS architecture.

The content being displayed may comprise at least one of a video, apicture, subtitle or the like, and the corresponding to-be-displayedmultimedia data thereof is respectively decoded video data, decodedpicture data, and subtitle. The video and the subtitle are all displayeddynamically, i.e., the content being displayed varies continuously inthe case of normal play without any operation and interference, thevideo stream may be divided into multiple graphic frames, the subtitlestream may be divided into multiple sentences, and each of the graphicframes and each sentence of the subtitle have a specified displayingperiod of time. However, the picture is displayed statically.

Complete displaying processes for different contents to be displayed areillustrated with reference to attached drawings hereinafter.

In a specific embodiment of the present disclosure, the content to bedisplayed is a video, and this embodiment is a further extension of theaforesaid embodiments and thus the same contents will not be repeatedherein. An overall displaying process of this embodiment is as shown inFIG. 12:

S101: setting a video file path to a play framework by the surfaceapplication program.

S102: acquiring by the surface application program from the playframework whether there is video stream data after the play frameworkparses the video file.

S103: creating a surface proxy body by the surface application programif there is video stream data.

S104: setting the surface proxy body to the play framework by thesurface application program.

S105: acquiring supported attribute information of a surface entrustingbody through the surface proxy body by the play framework, the supportedattribute information comprising a set of formats supported.

S106: configuring the surface entrusting body through the surface proxybody by the play framework.

For example, a format as well as a total number, width and height ofbuffers required are selected.

S107: acquiring a output buffer from the surface entrusting body throughthe surface proxy body by the play framework.

S108: filling the output buffer with to-be-displayed multimedia data bythe play framework.

The to-be-displayed multimedia data is decoded graphic frames to bedisplayed currently.

S109: sending the filled buffer to the surface entrusting body throughthe surface proxy body by the play framework.

S110: displaying the to-be-displayed multimedia data on a display screenby the surface entrusting body.

The buffer becomes free again after the displaying is finished.

S111: determining whether all the graphic frames have been displayed.

If the determination result is yes, then the process ends; and if thedetermination result is no, then the process skips to S107 and repeatsS107 and the subsequent steps.

In the specific displaying process, in addition to specific datarequired by the displaying (e.g., a handle of the surface proxy body,configuration parameters, buffer identifier (ID) or the like), controlcommands required for transmitting the specific data (e.g., instructionsof configuring the surface proxy body to the play framework,configuration instructions, instructions of acquiring the buffer or thelike) are also transmitted between the surface application program/playframework and the surface proxy body as well as between the surfaceproxy body and the surface entrusting body. The transmission of thespecific data and the control commands may be performed separately orperformed simultaneously. For example, during the process of configuringthe surface proxy body to the play framework by the surface applicationprogram, the surface application program sends the instruction ofconfiguring the surface proxy body and the handle of the surface proxybody to the play framework at the same time. In the displaying process,all the control commands may be called a control flow and all thespecific data may be called a data flow.

The following descriptions are based on whether the surface entrustingbody is the CS architecture and whether the object to be processed isthe control flow or data flow. In a specific process, if there is morethan one piece of data to be displayed, then after the current data hasbeen displayed, the part of calling the buffer, filling data and sendingdata to the surface realizing component at a bottom level for display isrepeatedly performed to continue to display the subsequent portion.

In the case where the surface entrusting body is the non-CS architectureand the object to be processed is the control flow, the displayingprocess is as shown in FIG. 13:

S121: creating a play framework by the surface application program.

S122: setting a multimedia file and loading the file by the surfaceapplication program.

S123: detecting whether there is a video stream by the surfaceapplication program.

S124: creating a surface proxy body by the surface application programif there is a video stream.

S125: setting a destination width, a destination height and adestination position to the surface proxy body by the surfaceapplication program.

S126: setting the surface proxy body to the play framework by thesurface application program.

S127: sending a request of acquiring a set of formats supported to thesurface proxy body by the play framework.

S128: sending the request of acquiring a set of formats supported to thesurface realization by the surface proxy body.

The play framework acquires a set of formats supported from the surfacerealization through the surface proxy body.

The play framework selects one of the set of formats supported as anoutput format.

S129: setting formats of the buffers, a the count of the buffers and theresolution to the surface proxy body by the play framework.

The format of the buffer matches with the output format.

S130: setting a surface realization according to the formats of thebuffers, the count of the buffers and the resolution by the surfaceproxy body.

S131: configuring the surface realizing component according to theformats of the buffers, the count of the buffers and the resolution bythe surface realization.

S132: notifying the surface realization to allocate the buffer by thesurface proxy body.

S133: allocating the buffer from the surface realizing component by thesurface realization.

S134: calling the surface proxy body to acquire one output buffer by theplay framework.

S315: filling the buffer with the to-be-displayed multimedia data by theplay framework.

S136: sending the buffer to the surface proxy body by the playframework.

S137: sending the buffer to the surface realization by the surface proxybody.

S138: sending the buffer to the surface realizing component by thesurface realization.

The surface realizing component displays the content in the buffer, andthe buffer become free again after the displaying is finished.

In the case where the surface entrusting body is the non-CS architectureand the object to be processed is the data flow, the displaying processis as shown in FIG. 14:

S141: sending a file path to the play framework by the surfaceapplication program.

S142: sending a the count and types of streams to the surfaceapplication program by the play framework after parsing the file.

The streams refer to media streams included in the file, e.g., videos,audios, subtitle or the like, and for example, 1 path of video, 2 pathof audio and 2 path of subtitle are obtained through the parsing.

S143: acquiring a handle of the surface proxy body by the surfaceapplication program.

S144: sending the handle of the surface proxy body to the play frameworkby the surface application program.

S145: sending the destination width, the destination height and thedestination position to the surface proxy body by the surfaceapplication program.

S146: sending a set of formats supported to the play framework by thesurface proxy body.

The play framework selects one from the set of formats supported as theoutput format.

S147: sending configuration information such as formats of the buffers,a the count of the buffers and the resolution or the like to the surfaceproxy body by the play framework.

S148: sending the configuration information such as the formats of thebuffers, the count of the buffers and the resolution or the like to thesurface realization by the surface proxy body.

S149: sending the configuration information such as the formats of thebuffers, the count of the buffers and the resolution or the like to thesurface realizing component by the surface realization.

S150: sending a set of free buffers acquired to the surface realizationby the surface realizing component.

S151: sending a output buffer acquired to the surface proxy body by thesurface realization.

The output buffer in this step belongs to the set of free buffers in theprevious step.

S152: sending the output buffer to the play framework by the surfaceproxy body.

S153: filling the output buffer with the to-be-displayed multimedia databy the play framework.

S154: sending the buffer filled with data to the surface proxy body bythe play framework.

S155: sending the buffer filled with data to the surface realization bythe surface proxy body.

S156: determining whether to display by the surface realization.

S157: sending the buffer filled with data to the surface realizingcomponent by the surface realization if it is determined to display.

The buffer returns back into the free status after the displaying isfinished.

S158: returning the output buffer acquired to the surface realization bythe surface realizing component after the displaying is finished.

In the case where the surface entrusting body is the CS architecture andthe object to be processed is the control flow, the displaying processis as shown in FIG. 15:

S161: creating a play framework by the surface application program.

S162: setting a multimedia file and loading the file by the surfaceapplication program.

S163: detecting whether there is a video stream by the surfaceapplication program.

S164: creating a surface proxy body by the surface application programif there is a video stream.

S165: creating a surface client by the surface proxy body.

S166: creating a corresponding surface object in a surface server by thesurface client.

S167: setting the surface proxy body to the play framework by thesurface application program.

S168: setting a destination width, a destination height and adestination position to the surface proxy body by the surfaceapplication program.

S169: setting the destination width, the destination height and thedestination position to the surface client by the surface proxy body.

S170: setting the destination width, the destination height and thedestination position to the surface object by the surface client.

S171: setting the destination width, the destination height and thedestination position to the surface realizing component by the surfaceserver.

S172: sending a request of acquiring a set of formats supported to thesurface proxy body by the play framework.

S173: sending the request of acquiring a set of formats supported to thesurface client by the surface proxy body.

S174: sending the request of acquiring a set of formats supported to thesurface server by the surface client.

S175: acquiring a set of formats supported from the surface realizingcomponent by the surface server.

S176: returning the set of formats supported to the play frameworkthrough the surface client and the surface proxy body by the surfaceserver.

The play framework selects one of the set of formats supported as anoutput format.

S177: setting configuration information such as formats of the buffers,a the count of the buffers and resolution or the like to the surfaceproxy body by the play framework.

S178: setting the configuration information such as the formats of thebuffers, the count of the buffers and the resolution or the like to thesurface client by the surface proxy body.

S179: setting the configuration information such as the formats of thebuffers, the count of the buffers and the resolution or the like to thesurface server by the surface client.

S180: setting the surface realizing component according to theconfiguration information such as the formats of the buffers, the countof the buffers and the resolution or the like by the surface server.

S181: notifying the surface server to allocate the buffer by the surfaceclient.

S182: allocating the buffer from the surface realizing component by thesurface server.

S183: calling the surface proxy body to send a request of acquiring onebuffer by the play framework.

S184: sending a request of acquiring one output buffer to the surfaceclient by the surface proxy body.

S185: sending the request of acquiring one output buffer to the surfaceserver by the surface client.

S186: acquiring one output buffer from the surface realizing componentby the surface server.

S187: returning the output buffer to the play framework through thesurface client and the surface proxy body by the surface server.

S188: filling the buffer with the to-be-displayed multimedia data by theplay framework.

S189: sending the buffer to the surface proxy body by the playframework.

S190: sending the buffer to the surface server by the surface proxybody.

S191: determining whether to display by the surface server.

S192: sending the buffer to the surface realizing component by thesurface server if it is determined to display.

In the case where the surface entrusting body is the CS architecture andthe object to be processed is the data flow, the displaying process isas shown in FIG. 16:

S201: sending a file path to the play framework by the surfaceapplication program.

S202: sending a the count and types of streams to the surfaceapplication program by the play framework after parsing the file.

S203: acquiring an ID of the surface object corresponding to the surfaceclient from the surface server by the surface client.

S204: acquiring a handle of the surface client by the surface proxybody.

S205: acquiring a handle of the surface proxy body by the surfaceapplication program.

S206: sending the handle of the surface proxy body to the play frameworkby the surface application program.

S207: sending the destination width, the destination height and thedestination position to the surface proxy body by the surfaceapplication program.

S208: acquiring a set of formats supported from the surface realizingcomponent by the surface server.

S209: acquiring the set of formats supported from the surface server bythe surface client.

S210: acquiring the set of formats supported from the surface client bythe surface proxy body.

S211: acquiring the set of formats supported from the surface proxy bodyby the play framework.

The play framework selects one of the set of formats supported as theoutput format.

S212: sending configuration information such as formats of the buffers,a the count of the buffers and resolution or the like to the surfaceproxy body by the play framework.

S213: sending the configuration information such as the formats of thebuffers, the count of the buffers and the resolution or the like to thesurface client by the surface proxy body.

S214: sending the configuration information such as the formats of thebuffers, the count of the buffers and the resolution or the like to thesurface server by the surface client.

S215: sending the configuration information such as the formats of thebuffers, the count of the buffers and the resolution or the like to thesurface realizing component by the surface server.

S216: sending a set of free buffers acquired to the surface server bythe surface realizing component.

S217: sending the set of free buffers acquired to the surface client bythe surface server.

S218: sending one output buffer acquired to the surface proxy body bythe surface client.

The output buffer belongs to the set of free buffers in the previousstep.

S219: sending the output buffer to the play framework by the surfaceproxy body.

S220: filling the output buffer with the to-be-displayed multimedia databy the play framework.

S221: sending the buffer filled with data to the surface proxy body bythe play framework.

S222: sending the buffer filled with data to the surface client by thesurface proxy body.

S223: sending the buffer filled with data to the surface server by thesurface client.

S224: sending the buffer filled with data to the surface realizingcomponent by the surface server.

The buffer returns back into the free status after the displaying isfinished.

S225: acquiring the output buffer by the surface realizing component.

In another specific embodiment of the present disclosure, the content tobe displayed is subtitle and the display entrusting side calls afunction provided by the surface application program to display thesubtitle, and this embodiment is a further extension of the aforesaidembodiments and thus the same contents will not be repeated herein. Anoverall displaying process of this embodiment is as shown in FIG. 17:

S231: setting a file path to the play framework by the surfaceapplication program.

S232: acquiring by the surface application program from the playframework whether there is subtitle stream data after the play frameworkparses the file.

S233: creating the surface proxy body by the surface application programif there is subtitle stream data.

S234: setting the surface proxy body to the play framework by thesurface application program.

S235: acquiring supported attribute information of the surfaceentrusting body through the surface proxy body by the play framework,the supported attribute information comprising a set of formatssupported and/or a set of characters supported or the like.

S236: configuring the surface entrusting body through the surface proxybody by the play framework.

For example, a format as well as a total number, width and height ofbuffers required or the like are selected.

S237: acquiring a output buffer from the surface entrusting body throughthe surface proxy body by the play framework.

S238: filling the output buffer with to-be-displayed multimedia data bythe play framework.

The to-be-displayed multimedia data is the current subtitle, and it maybe in the picture form or in the text form after character setconversion.

S239: sending the filled buffer to the surface entrusting body throughthe surface proxy body by the play framework.

S240: calling a function provided by the surface application program toplot the current subtitle in the picture or text form by the surfaceentrusting body.

If the surface entrusting body is the non-CS architecture, then thesurface realization receives the filled buffer from the surface proxybody and then calls the function to plot the current subtitle. If thesurface entrusting body is the CS architecture, then the surface clientreceives the filled buffer from the surface proxy body and then sendsthe filled buffer to the surface server, and the surface server callsthe function to plot the current subtitle.

The buffer becomes free again after the displaying is finished.

S241: determining whether all the subtitle has been displayed.

If the determination result is yes, then the process ends; and if thedetermination result is no, then the process skips to S237 and repeatsS237 and the subsequent steps.

The following descriptions take the surface entrusting body of a non-CSarchitecture as an example and are based on whether the object to beprocessed is the control flow or data flow. In a specific process, ifthere is more than one piece of data to be displayed, then after thecurrent data has been displayed, the part of calling the buffer, fillingdata and sending data to the surface realizing component at a bottomlevel for display may be repeatedly performed to continue to display thesubsequent portion.

In the case where the object to be processed is the control flow, thedisplaying process is as shown in FIG. 18:

S251: creating a play framework by the surface application program.

S252: setting a multimedia file and loading the file by the surfaceapplication program.

S253: detecting whether there is a subtitle stream by the surfaceapplication program.

S254: creating the surface proxy body by the surface application programif there is a subtitle stream.

S255: setting the surface proxy body to the play framework by thesurface application program.

S256: sending a request of acquiring a set of formats supported and aset of characters supported by draw text to the surface proxy body bythe play framework.

draw text is a plotting function provided by the surface applicationprogram that is capable of processing subtitle in the text form.

S257: sending a request of acquiring a set of formats supported by thesurface realization and the set of characters supported by draw text tothe surface realization by the surface proxy body.

S258: acquiring a set of formats supported by draw picture of thesurface application program and the set of characters supported by drawtext by the surface realization.

draw picture is a plotting function provided by the surface applicationprogram that is capable of processing subtitle in the picture form.

The play framework acquires the set of formats supported and the set ofcharacters supported from the surface realization through the surfaceproxy body.

The play framework selects one of the set of formats supported/the setof characters supported as an output format.

S259: setting formats of the buffers (in the case of picture subtitle),a the count of the buffers and resolution to the surface proxy body bythe play framework.

The format of the buffer matches with the output format.

S260: setting the surface realization according to the formats of thebuffers, the count of the buffers and the resolution by the surfaceproxy body.

S261: allocating the buffer by the surface proxy body.

S262: calling the surface proxy body to acquire one buffer by the playframework.

S263: converting the subtitle into a set of characters supported by theplay framework in the case of the text subtitle.

S264: filling the buffer with the to-be-displayed multimedia data by theplay framework.

S265: sending the buffer to the surface proxy body by the playframework.

S266: sending the buffer to the surface realization by the surface proxybody.

S267: calling a function provided by the surface application program toplot the subtitle picture or text by the surface realization.

In the case where the object to be processed is the data flow, thedisplaying process is as shown in FIG. 19:

S271: sending a file path to the play framework by the surfaceapplication program.

S272: sending a the count and types of streams to the surfaceapplication program by the play framework after parsing the file.

The streams refer to media streams included in the file, e.g., videos,audios, subtitle or the like, and for example, 1 path of video, 2 pathof audio and 2 path of subtitle are obtained through the parsing.

S273: acquiring a handle of the surface proxy body by the surfaceapplication program.

S274: sending the handle of the surface proxy body to the play frameworkby the surface application program.

S275: sending a request of acquiring a set of formats supported and aset of characters supported to the surface realization through thesurface proxy body by the surface application program.

S276: sending a set of formats supported and a set of characterssupported to the surface proxy body by the surface realization.

S277: sending the set of formats supported and the set of characterssupported to the play frame by the surface proxy body.

The play framework selects one from the set of formats supported and theset of characters supported as the output format.

S278: sending configuration information such as formats of the buffers,a the count of the buffers and resolution or the like to the surfaceproxy body by the play framework.

S279: sending the configuration information such as the formats of thebuffers, the count of the buffers and the resolution or the like to thesurface realization by the surface proxy body.

S280: sending one output buffer acquired to the surface proxy body bythe surface realization.

S281: sending the output buffer to the play framework by the surfaceproxy body.

S282: filling the output buffer with the to-be-displayed multimedia databy the play framework.

S283: sending the buffer filled with data to the surface proxy body bythe play framework.

S284: sending the buffer filled with data to the surface realization bythe surface proxy body.

S285: sending picture data or text data that needs to be plotted to thesurface application program for plotting by the surface realization.

The surface entrusting body is the non-CS architecture in all of thespecific processes shown in this embodiment, and if the surfaceentrusting body is the CS architecture, then the surface server thereincalls the function provided by the surface application program to plotthe subtitle.

In another specific embodiment of the present disclosure, the content tobe displayed is subtitle and the display entrusting side calls thesurface realizing component to display the subtitle, and this embodimentis a further extension of the aforesaid embodiments and thus the samecontents will not be repeated herein. An overall displaying process ofthis embodiment is as shown in FIG. 20:

S301: setting a file path to the play framework by the surfaceapplication program.

S302: acquiring by the surface application program from the playframework whether there is subtitle stream data after the play frameworkparses the file.

S303: creating the surface proxy body by the surface application programif there is subtitle stream data.

S304: setting the surface proxy body to the play framework by thesurface application program.

S305: acquiring supported attribute information of the surfaceentrusting body through the surface proxy body by the play framework,the supported attribute information comprising a set of formatssupported and/or a set of characters supported or the like.

S306: configuring the surface entrusting body through the surface proxybody by the play framework.

For example, a format as well as a total number, width and height ofbuffers required or the like are selected.

S307: acquiring a output buffer from the surface entrusting body throughthe surface proxy body by the play framework.

S308: if the subtitle to be displayed is in the text form, thenconverting the subtitle into picture data by the play framework.

If the subtitle to be displayed is in the picture form, then this stepmay be omitted.

S309: filling the output buffer with to-be-displayed multimedia data bythe play framework.

The to-be-displayed multimedia data is the subtitle data in the pictureform.

S310: sending the filled buffer to the surface entrusting body throughthe surface proxy body by the play framework.

S311: displaying the to-be-displayed multimedia data on a display screenby the surface entrusting body.

The buffer becomes free again after the displaying is finished.

S312: determining whether all the subtitle has been displayed.

If the determination result is yes, then the process ends; and if thedetermination result is no, then the process skips to S307 and repeatsS307 and the subsequent steps.

For the specific process in the cases whether the surface entrustingbody is the CS architecture and the object to be processed is thecontrol flow or data flow, except for that the play framework needs todetermine whether the subtitle to be displayed is in the text formbefore filling the buffer with the to-be-displayed multimedia data andconverts the subtitle into picture form if the subtitle is in the textform, reference may be made to the specific processing flows of videosdescribed above for the remaining part, and thus the remaining part willnot be repeated herein.

In yet another specific embodiment of the present disclosure, thecontent to be displayed is a picture, and this embodiment is a furtherextension of the aforesaid embodiments and thus the same contents willnot be repeated herein. An overall displaying process of this embodimentis as shown in FIG. 21:

S401: creating a surface proxy body by the surface application program.

S402: setting configuration information such as a destination width, adestination height, a destination position or the like to the surfaceproxy body by the surface application program.

S403: setting a to-be-decoded picture file path or picture data to apicture display framework by the surface application program.

S404: decoding the picture file or picture data by the picture displayframework.

S405: acquiring the decoded data and information such as the width,height and format of the decoded data or the like from the picturedisplay framework by the surface application program.

S406: sending the decoded data and the information such as the width,height and format of the decoded data or the like to the surface proxybody by the surface application program.

S407: sending the decoded data and the information such as the width,height and format of the decoded data or the like to the surfaceentrusting body by the surface proxy body.

S408: displaying the to-be-displayed multimedia data on the displayscreen by the surface entrusting body.

The following descriptions are based on whether the surface entrustingbody is the CS architecture and whether the object to be processed isthe control flow or data flow.

In the case where the surface entrusting body is the non-CS architectureand the object to be processed is the control flow, the displayingprocess is as shown in FIG. 22:

S411: creating the surface proxy body by the surface applicationprogram.

S412: creating, by the surface proxy body, a handle corresponding to thesurface proxy body itself.

S413: setting a picture display position and a destination size to thesurface proxy body by the surface application program.

S414: setting the picture display position and the destination size tothe surface realization by the surface proxy body.

S415: setting a to-be-decoded picture file path or picture data to thepicture display framework by the surface application program.

S416: acquiring the decoded to-be-displayed multimedia data andinformation such as the format, width and height corresponding to theto-be-displayed multimedia data or the like from the picture displayframework by the surface application program.

S417: sending the decoded to-be-displayed multimedia data and theinformation such as the width, height and format corresponding to theto-be-displayed multimedia data or the like to the surface proxy body bythe surface application program.

S418: sending the decoded to-be-displayed multimedia data and theinformation such as the corresponding width, height and format as wellas the destination width and height and the destination position or thelike to the surface realization by the surface proxy body.

S419: determining whether to display by the surface realization.

S420: if it is determined to display, then sending the decodedto-be-displayed multimedia data and the information such as thecorresponding width, height and format as well as the destination widthand height and the destination position or the like to the surfacerealizing component for display by the surface realization.

In the case where the surface entrusting body is the non-CS architectureand the object to be processed is the data flow, the displaying processis as shown in FIG. 23:

S421: sending a handle of the surface realization to the surface proxybody by the surface realization.

S422: sending the handle of the surface proxy body to the surfaceapplication program by the surface proxy body.

S423: setting the picture display position and the destination size tothe surface proxy body by the surface application program.

S424: setting the picture display position and the destination size tothe surface realization by the surface proxy body.

S425: setting the to-be-decoded picture file path or picture data to thepicture display framework by the surface application program.

S426: acquiring the decoded to-be-displayed multimedia data andinformation such as the format, width and height corresponding to theto-be-displayed multimedia data or the like from the picture displayframework by the surface application program.

S427: sending the decoded to-be-displayed multimedia data and theinformation such as the width, height and format corresponding to theto-be-displayed multimedia data or the like to the surface proxy body bythe surface application program.

S428: sending the decoded to-be-displayed multimedia data and theinformation such as the corresponding width, height and format as wellas the destination width and height and the destination position or thelike to the surface realization by the surface proxy body.

S429: sending the decoded to-be-displayed multimedia data and theinformation such as the corresponding width, height and format as wellas the destination width and height and the destination position or thelike to the surface realizing component for display by the surfacerealization.

In the case where the surface entrusting body is the CS architecture andthe object to be processed is the control flow, the displaying processis as shown in FIG. 24:

S431: creating the surface proxy body by the surface applicationprogram.

S432: creating a corresponding surface client by the surface proxy body.

S433: creating a corresponding surface object in the surface server bythe surface client.

S434: setting the picture display position and the destination size tothe surface proxy body by the surface application program.

S435: setting the picture display position and the destination size tothe surface client by the surface proxy body.

S436: setting a to-be-decoded picture file path or picture data to thepicture display framework by the surface application program.

S437: acquiring the decoded to-be-displayed multimedia data andinformation such as the format, width and height corresponding to theto-be-displayed multimedia data or the like from the picture displayframework by the surface application program.

S438: sending the decoded to-be-displayed multimedia data and theinformation such as the width, height and format corresponding to theto-be-displayed multimedia data or the like to the surface proxy body bythe surface application program.

S439: sending the decoded to-be-displayed multimedia data and theinformation such as the corresponding width, height and format as wellas the destination width and height and the destination position or thelike to the surface client by the surface proxy body.

S440: sending the decoded to-be-displayed multimedia data and theinformation such as the corresponding width, height and format as wellas the destination width and height and the destination position or thelike to the surface server by the surface client.

S441: determining whether to display by the surface server.

S442: if it is determined to display, then sending the decodedto-be-displayed multimedia data and the information such as thecorresponding width, height and format as well as the destination widthand height and the destination position or the like to the surfacerealizing component for display by the surface server.

In the case where the surface entrusting body is the CS architecture andthe object to be processed is the data flow, the displaying process isas shown in FIG. 25:

S451: sending the ID of the surface object corresponding to the surfaceclient to the surface client by the surface server.

S452: sending a handle of the surface client to the surface proxy bodyby the surface client.

S453: sending the handle of the surface proxy body to the surfaceapplication program by the surface proxy body.

S454: setting the picture display position and the destination size tothe surface proxy body by the surface application program.

S455: setting the picture display position and the destination size tothe surface client by the surface proxy body.

S456: setting the to-be-decoded picture file path or picture data to thepicture display framework by the surface application program.

S457: acquiring the decoded to-be-displayed multimedia data andinformation such as the format, width and height corresponding to theto-be-displayed multimedia data or the like from the picture displayframework by the surface application program.

S458: sending the decoded to-be-displayed multimedia data and theinformation such as the width, height and format corresponding to theto-be-displayed multimedia data or the like to the surface proxy body bythe surface application program.

S459: sending the decoded to-be-displayed multimedia data and theinformation such as the corresponding width, height and format as wellas the destination width and height and the destination position or thelike to the surface client by the surface proxy body.

S460: sending the decoded to-be-displayed multimedia data and theinformation such as the corresponding width, height and format as wellas the destination width and height and the destination position or thelike to the surface server by the surface client.

S461: sending the decoded to-be-displayed multimedia data and theinformation such as the corresponding width, height and format as wellas the destination width and height and the destination position or thelike to the surface realizing component for display by the surfaceserver.

As shown in FIG. 26, an embodiment of a proxy-pattern-based multimediadisplaying device of the present disclosure is a chip where an embeddedsystem is built in, the embedded system comprises a display applicationlayer, a surface proxy body, a surface entrusting body and a surfacerealizing component, and the multimedia displaying device comprises:

a surface proxy body 51, being configured to receive to-be-displayedmultimedia data from the display application layer and send theto-be-displayed multimedia data to a surface entrusting body 52; and

the surface entrusting body 52, being configured to send theto-be-displayed multimedia data to the surface realizing component orthe display application layer for display on a display screen.

The surface proxy body 51 is a proxy for the surface entrusting body 52,and the surface proxy body 51 and the surface entrusting body 52 areindependent of the display application layer.

Optionally, the surface entrusting body 52 is of a non-CS architectureor a CS architecture. In case of the non-CS architecture, the surfaceentrusting body 52 comprises a surface realization which interfaces withthe surface realizing component or the display application layer torealize the display function. In case of the CS architecture, thesurface entrusting body 52 comprises a surface client and a surfaceserver, and the surface server interfaces with the surface realizingcomponent or the display application layer to realize the displayfunction.

Optionally, the to-be-displayed multimedia data comprises at least oneof decoded picture data, decoded video data and subtitle. The surfaceproxy body 51 is specifically configured to: acquire a output bufferfrom the surface entrusting body 52; send the output buffer to thedisplay application layer so that the display application layer fillsthe output buffer with the to-be-displayed multimedia data; receive thefilled buffer from the display application layer; and send the filledbuffer to the surface realization or the surface client. If the surfaceentrusting body 52 is of the non-CS architecture, then the surfacerealization is specifically configured to: send the filled buffer to thesurface realizing component so that the surface realizing componentcontrols the display screen to display the to-be-displayed multimediadata. If the surface entrusting body 52 is of the CS architecture, thenthe surface client is specifically configured to send the filled bufferto the surface server; and the surface server is specifically configuredto send the filled buffer to the surface realizing component so that thesurface realizing component controls the display screen to display theto-be-displayed multimedia data.

Optionally, the to-be-displayed multimedia data comprises subtitle. Thesurface proxy body 51 is specifically configured to: acquire a outputbuffer from the surface entrusting body 52; send the output buffer tothe display application layer so that the display application layerfills the output buffer with the subtitle; receive the filled bufferfrom the display application layer; and send the filled buffer to thesurface realization or the surface client. If the surface entrustingbody 52 is of the non-CS architecture, then the surface realization isspecifically configured to call a function provided by the displayapplication layer to plot the subtitle on the display screen. If thesurface entrusting body 52 is of the CS architecture, then the surfaceclient is specifically configured to send the filled buffer to thesurface server; and the surface server is specifically configured tocall a function provided by the display application layer to plot thesubtitle on the display screen.

Optionally, the surface entrusting body 52 is of the CS architecture,the display application layer comprises at least one surface applicationprogram, each of the at least one surface application programcorresponds to one surface proxy body 51 and one surface clientrespectively, each surface client corresponds to one surface objectrespectively in the surface server and the surface server manages thesurface object.

If the number of the at least one surface application program is greaterthan one, then the surface server is specifically configured to: mix theto-be-displayed multimedia data from at least two of the surfaceapplication programs and then send the mixed to-be-displayed multimediadata to the surface realizing component; or determine whether the numberof the display screen satisfies the displaying need of the surfaceapplication program, and if the determination result is that the numberof the display screen does not satisfy the displaying need of thesurface application program, then send the to-be-displayed multimediadata corresponding to part of the surface application programs to thesurface realizing component; and if the determination result is that thenumber of the display screen satisfies the displaying need of thesurface application program, then send all of the to-be-displayedmultimedia data to the surface realizing component.

If the number of the display screen is greater than one, each of thedisplay screens corresponds to one coverage object respectively in thesurface realizing component; and the surface server is specificallyconfigured to copy the to-be-displayed multimedia data and then send thecopied to-be-displayed multimedia data to at least two of the coverageobjects.

Optionally, the surface proxy body 51 is further configured to: receivefrom the display application layer a request of acquiring supportedattribute information of the proxy entrusting body, the supportedattribute information comprising a set of formats supported and/or a setof characters supported; forward the request of acquiring the supportedattribute information of the proxy entrusting body to the surfaceentrusting body 52; and receive the supported attribute information fromthe surface entrusting body 52 and send the supported attributeinformation to the display application layer.

Optionally, the surface proxy body 51 is further configured to receiveconfiguration information from the display application layer and sendthe configuration information to the surface entrusting body 52, and theconfiguration information comprises at least one of a selected format,resolution, buffer information, destination information and priority.

As shown in FIG. 27, an embodiment of a proxy-pattern-based multimediadisplaying apparatus of the present disclosure comprises a processor 610and a storage 620, and the processor 610 is connected to the storage620.

The storage 620 is configured to store instructions used for theoperation of the processor 610.

The processor 610 controls the operation of the proxy-pattern-basedmultimedia displaying device, and the processor 610 may also be referredto as a central processing unit (CPU). The processor 610 may be anintegrated circuit chip with the signal processing capability. Theprocessor 610 may also be a universal processor, a digital signalprocessor (DSP), an application-specific integrated circuit (ASIC), afield programmable gate array (FPGA) or other programmable logicdevices, a discrete gate or a transistor logic device and a discretehardware component. The universal processor may be a microprocessor orthe processor may be any conventional processor or the like.

The storage 620 is configured to store instructions used for theoperation of the processor 610.

The processor 610 is configured to execute the instructions toaccomplish the method provided in any one and any non-conflictingcombination of the embodiments of the proxy-pattern-based multimediadisplaying method according to the present disclosure.

As shown in FIG. 28, an embodiment of a readable storage medium of thepresent disclosure comprises a storage 710 having instructions storedthereon, and when the instructions are executed, the method provided inany one and any non-conflicting combination of the embodiments of theproxy-pattern-based multimedia displaying method according to thepresent disclosure is accomplished.

The storage 710 may comprise a read-only memory (ROM), a random accessmemory (RAM), a flash memory, a hard disk, a compact disk or the like.

It shall be appreciated that, methods and devices disclosed in theembodiments provided in the present disclosure may be accomplished inother ways. For example, the implementation of the device describedabove is only schematic, e.g., the division of the modules or units isonly a division in terms of logic functions, but other dividing mannersare possible in the practical implementation. For example, multipleunits or components may be combined or integrated into another system,or some features may be ignored or not executed. Additionally, thecoupling or direct coupling or communication connection between thecomponents that are shown or discussed may be accomplished by someinterfaces, and the indirect coupling or communication connectionbetween devices or units may be electrical, mechanical or in otherforms.

A unit that is illustrated as a separate part may be or may not bephysically separated, and a part that is displayed as a unit may be ormay not be a physical unit, i.e., the part may be located at one placeor may be distributed over multiple network units. The objective of thesolution of this implementation can be achieved by selecting part or allof the units depending on practical needs.

Moreover, the functional units in the embodiments of the presentdisclosure may be integrated into one processing unit, or each of theunits exists separately as a physical unit, or two or more of the unitsare integrated into one unit. The integrated unit described above may beimplemented in the form of hardware or may be implemented in the form ofa software functional unit.

The integrated unit, if implemented in the form of a software functionalunit and sold or used as an independent product, may be stored in onecomputer readable storage medium. Based on such understanding, thetechnical solution of the present disclosure may be substantiallyembodied in the form of a software product or part of the technicalsolution that contributes to the prior art or all or part of thetechnical solution may be embodied in the form of a software product.The computer software product is stored in one storage medium andcomprises multiple instructions to enable a computer apparatus (whichmay be a personal computer, a server, or a network apparatus or thelike) or a processor to execute all or part of the steps of the methodsof the embodiments according to the present disclosure. the storagemedium described above includes various mediums that are capable ofstoring program codes such as a USB flash disk, a mobile hard disk, aread-only memory (ROM), a random access memory (RAM), a magnetic disk ora compact disk or the like.

What described above are only the embodiments of the present disclosure,but are not intended to limit the scope of the present disclosure. Anyequivalent structures or equivalent process flow modifications that aremade according to the specification and the attached drawings of thepresent disclosure, or any direct or indirect applications of thepresent disclosure in other related technical fields shall all becovered within the scope of the present disclosure.

What is claimed is:
 1. A proxy-pattern-based multimedia displayingmethod applied to a chip where an embedded system is built in, theembedded system comprising a display application layer, a surface proxybody, a surface entrusting body and a surface realizing component,wherein the surface proxy body is an proxy for the surface entrustingbody and the surface proxy body and the surface entrusting body areindependent of the display application layer; the multimedia displayingmethod comprising: receiving to-be-displayed multimedia data from thedisplay application layer and sending the to-be-displayed multimediadata to the surface entrusting body by the surface proxy body; andsending the to-be-displayed multimedia data to the surface realizingcomponent or the display application layer for display on a displayscreen by the surface entrusting body.
 2. The method of claim 1, whereinthe surface entrusting body is one of a non-CS architecture and a CSarchitecture; when the surface entrusting body is the non-CSarchitecture, the surface entrusting body comprising a surfacerealization which interfaces with one of the surface realizing componentand the display application layer to realize a display function; whenthe surface entrusting body is the CS architecture, the surfaceentrusting body comprising a surface client and a surface server, andthe surface server interfaces with one of the surface realizingcomponent and the display application layer to realize the displayfunction.
 3. The method of claim 2, wherein the to-be-displayedmultimedia data comprises at least one of decoded picture data, decodedvideo data and subtitle; the receiving to-be-displayed multimedia datafrom the display application layer and sending the to-be-displayedmultimedia data to the surface entrusting body by the surface proxy bodycomprising: acquiring an output buffer from the surface entrusting bodyby the surface proxy body; sending the output buffer to the displayapplication layer so that the display application layer fills the outputbuffer with the to-be-displayed multimedia data by the surface proxybody; receiving the filled buffer from the display application layer bythe surface proxy body; and sending the filled buffer to one of thesurface realization and the surface client by the surface proxy body. 4.The method of claim 3, wherein the to-be-displayed multimedia datacomprises the subtitle in a text format, and the subtitle data in thefilled buffer is subtitle in a picture format that has been converted bythe display application layer.
 5. The method of claim 3, wherein thesurface entrusting body is the non-CS architecture, and the sending theto-be-displayed multimedia data to the surface realizing component fordisplay on the display screen by the surface entrusting body comprises:sending the filled buffer to the surface realizing component by thesurface realization so that the surface realizing component controls thedisplay screen to display the to-be-displayed multimedia data; or thesurface entrusting body is the CS architecture, and the sending theto-be-displayed multimedia data to surface realizing component fordisplay on the display screen by the surface entrusting body comprises:sending the filled buffer to the surface server by the surface client;and sending the filled buffer to the surface realizing component by thesurface server so that the surface realizing component controls thedisplay screen to display the to-be-displayed multimedia data.
 6. Themethod of claim 2, wherein the to-be-displayed multimedia data comprisessubtitle, the surface entrusting body is the non-CS architecture, andthe receiving the to-be-displayed multimedia data from the displayapplication layer and sending the to-be-displayed multimedia data to thesurface entrusting body by the surface proxy body comprises: acquiringan output buffer from the surface entrusting body by the surface proxybody; sending the output buffer to the display application layer so thatthe display application layer fills the output buffer with the subtitleby the surface proxy body; receiving the filled buffer from the displayapplication layer by the surface proxy body; and sending the filledbuffer to the surface realization by the surface proxy body; and thesending the to-be-displayed multimedia data to the display applicationlayer for display on the display screen by the surface entrusting bodycomprises: calling a function provided by the display application layerto plot the subtitle on the display screen by the surface realization.7. The method of claim 2, wherein the to-be-displayed multimedia datacomprises subtitle, the surface entrusting body is the CS architecture;the receiving the to-be-displayed multimedia data from the displayapplication layer and sending the to-be-displayed multimedia data to thesurface entrusting body by the surface proxy body comprises: acquiringan output buffer from the surface entrusting body by the surface proxybody; sending the output buffer to the display application layer by thesurface proxy body so that the display application layer fills theoutput buffer with the to-be-displayed multimedia data; receiving thefilled buffer from the display application layer by the surface proxybody; and sending the filled buffer to the surface client by the surfaceproxy body; and the sending the to-be-displayed multimedia data tosurface realizing component for display on the display screen by thesurface entrusting body comprises: sending the filled buffer to thesurface server by the surface client; and calling by the surface servera function provided by the display application layer to plot thesubtitle on the display screen.
 8. The method of claim 2, wherein thesurface entrusting body is the CS architecture, the display applicationlayer comprises at least one surface application program, each of the atleast one surface application program corresponds to one said surfaceproxy body and one said surface client respectively, each said surfaceclient corresponds to one surface object respectively in the surfaceserver and the surface server manages the surface object.
 9. The methodof claim 8, wherein the number of the at least one surface applicationprogram is greater than one, sending by the surface entrusting body theto-be-displayed multimedia data to the surface realizing componentcomprising: mixing the to-be-displayed multimedia data from at least twoof the surface application programs and then sending the mixedto-be-displayed multimedia data to the surface realizing component bythe surface server; or determining by the surface server whether thenumber of the display screen satisfies the displaying need of thesurface application program, and when the determination result is thatthe number of the display screen does not satisfy the displaying need ofthe surface application program, then sending by the surface server theto-be-displayed multimedia data corresponding to part of the surfaceapplication programs to the surface realizing component, and when thedetermination result is that the number of the display screen satisfiesthe displaying need of the surface application program, then sending bythe surface server all of the to-be-displayed multimedia data to thesurface realizing component.
 10. The method of claim 8, wherein thenumber of the display screen is greater than one, and each displayscreen corresponds to one coverage object respectively in the surfacerealizing component; sending the to-be-displayed multimedia data to thesurface realizing component by the surface entrusting body comprising:copying the to-be-displayed multimedia data and then sending the copiedto-be-displayed multimedia data to at least two of the coverage objectsby the surface server.
 11. The method of claim 1, before the receivingby the surface proxy body the to-be-displayed multimedia data from thedisplay application layer further comprising: receiving by the surfaceproxy body from the display application layer a request of acquiringsupported attribute information of the proxy entrusting body, thesupported attribute information comprising a set of formats supportedand/or a set of characters supported; forwarding by the surface proxybody the request of acquiring the supported attribute information of theproxy entrusting body to the surface entrusting body; sending by thesurface entrusting body the supported attribute information to thesurface proxy body; sending by the surface proxy body the supportedattribute information to the display application layer: receiving by thesurface proxy body configuration information from the displayapplication layer, the configuration information comprising at least oneof a selected format, resolution, buffer information, destinationinformation and priority; and sending by the surface proxy body theconfiguration information to the surface entrusting body.
 12. Aproxy-pattern-based multimedia displaying device belonged to to a chipwhere an embedded system is built in, the embedded system comprising adisplay application layer, a surface proxy body, a surface entrustingbody and a surface realizing component, the multimedia displaying methodcomprising: the surface proxy body configured to receive to-be-displayedmultimedia data from the display application layer and send theto-be-displayed multimedia data to the surface entrusting body; and thesurface entrusting body configured to send the to-be-displayedmultimedia data to the surface realizing component or the displayapplication layer for display on a display screen; wherein the surfaceproxy body is an proxy for the surface entrusting body and the surfaceproxy body and the surface entrusting body are independent of thedisplay application layer.
 13. The device of claim 12, wherein thesurface entrusting body is one of a non-CS architecture and a CSarchitecture; when the surface entrusting body is the non-CSarchitecture, the surface entrusting body comprising a surfacerealization which interfaces with one of the surface realizing componentand the display application layer to realize a display function; whenthe surface entrusting body is the CS architecture, the surfaceentrusting body comprising a surface client and a surface server, andthe surface server interfaces with one of the surface realizingcomponent and the display application layer to realize the displayfunction.
 14. The device of claim 13, wherein the to-be-displayedmultimedia data comprising at least one of decoded picture data, decodedvideo data and subtitle; the surface proxy body is configured to acquirea output buffer from the surface entrusting body; send the output bufferto the display application layer so that the display application layerfills the output buffer with the to-be-displayed multimedia data;receive the filled buffer from the display application layer; and s oneof the surface realization and the surface client; and the surfaceentrusting body is the non-CS architecture, and the surface realizationis configured to send the filled buffer to the surface realizingcomponent so that the surface realizing component controls the displayscreen to display the to-be-displayed multimedia data; or the surfaceentrusting body is the CS architecture, the surface client is configuredto send the filled buffer to the surface server; and the surface serveris configured to send the filled buffer to the surface realizingcomponent so that the surface realizing component controls the displayscreen to display the to-be-displayed multimedia data.
 15. The device ofclaim 13, wherein the to-be-displayed multimedia data comprisessubtitle; the surface proxy body is configured to acquire a outputbuffer from the surface entrusting body; send the output buffer to thedisplay application layer so that the display application layer fillsthe output buffer with the subtitle; receive the filled buffer from thedisplay application layer; and send the filled buffer to the surfacerealization or the surface client; the surface entrusting body is thenon-CS architecture, and the surface realization is configured to call afunction provided by the display application layer to plot the subtitleon the display screen; or the surface entrusting body is the CSarchitecture, and the surface client is configured to send the filledbuffer to the surface server; and the surface server is configured tocall a function provided by the display application layer to plot thesubtitle on the display screen.
 16. The device of claim 13, wherein thesurface entrusting body is the CS architecture, the display applicationlayer comprising at least one surface application program, each of theat least one surface application program corresponds to one said surfaceproxy body and one said surface client respectively, each said surfaceclient corresponds to one surface object respectively in the surfaceserver and the surface server manages the surface object.
 17. The deviceof claim 16, wherein the number of the at least one surface applicationprogram is greater than one, and the surface server is configured to:mix the to-be-displayed multimedia data from at least two of the surfaceapplication programs and then send the mixed to-be-displayed multimediadata to the surface realizing component; or determine whether the numberof the display screen satisfies the displaying need of the surfaceapplication program, and when the determination result is that thenumber of the display screen does not satisfy the displaying need of thesurface application program, then send the to-be-displayed multimediadata corresponding to part of the surface application programs to thesurface realizing component, and when the determination result is thatthe number of the display screen satisfies the displaying need of thesurface application program, then send all of the to-be-displayedmultimedia data to the surface realizing component.
 18. The device ofclaim 16, wherein the number of the display screen is greater than one,and each of the display screens corresponds to one coverage objectrespectively in the surface realizing component; and the surface serveris configured to copy the to-be-displayed multimedia data and then sendthe copied to-be-displayed multimedia data to at least two of thecoverage objects.
 19. A proxy-pattern-based multimedia displayingapparatus comprising a processor and a storage, the processor beingconnected to the storage and the storage having instructions storedtherein; the processor is configured to execute the instructions toaccomplish a proxy-pattern-based multimedia displaying method applied toa chip where an embedded system is built in, the embedded systemcomprising a display application layer, a surface proxy body, a surfaceentrusting body and a surface realizing component, wherein the surfaceproxy body is an proxy for the surface entrusting body and the surfaceproxy body and the surface entrusting body are independent of thedisplay application layer; the multimedia displaying method comprising:receiving to-be-displayed multimedia data from the display applicationlayer and sending the to-be-displayed multimedia data to the surfaceentrusting body by the surface proxy body; and sending theto-be-displayed multimedia data to the surface realizing component orthe display application layer for display on a display screen by thesurface entrusting body.